Approximate Inference and Constrained Optimization
نویسندگان
چکیده
Loopy and generalized belief propagation are popular algorithms for approximate inference in Markov random fields and Bayesian networks. Fixed points of these algorithms correspond to extrema of the Bethe and Kikuchi free energy (Yedidia et al., 2001). However, belief propagation does not always converge, which motivates approaches that explicitly minimize the Kikuchi/Bethe free energy, such as CCCP (Yuille, 2002) and UPS (Teh and Welling, 2002). Here we describe a class of algorithms that solves this typically non-convex constrained minimization problem through a sequence of convex constrained minimizations of upper bounds on the Kikuchi free energy. Intuitively one would expect tighter bounds to lead to faster algorithms, which is indeed convincingly demonstrated in our simulations. Several ideas are applied to obtain tight convex bounds that yield dramatic speed-ups over CCCP.
منابع مشابه
A semidefinite relaxation scheme for quadratically constrained
Semidefinite optimization relaxations are among the widely used approaches to find global optimal or approximate solutions for many nonconvex problems. Here, we consider a specific quadratically constrained quadratic problem with an additional linear constraint. We prove that under certain conditions the semidefinite relaxation approach enables us to find a global optimal solution of the unde...
متن کاملSequential Optimality Conditions and Variational Inequalities
In recent years, sequential optimality conditions are frequently used for convergence of iterative methods to solve nonlinear constrained optimization problems. The sequential optimality conditions do not require any of the constraint qualications. In this paper, We present the necessary sequential complementary approximate Karush Kuhn Tucker (CAKKT) condition for a point to be a solution of a ...
متن کاملConstrained Kronecker Deltas for Fast Approximate Inference and Estimation
We are interested in constructing fast, principled algorithms for scaling up decoding and parameter estimation in probability models such as Hidden Markov Models (HMMs) and linear-chain Conditional Random Fields (CRFs) with large state spaces. In this paper we present a principled extension of beam search to beam inference and learning. We present a new approach for approximate inference based ...
متن کاملADAPTIVE NEURO-FUZZY INFERENCE SYSTEM OPTIMIZATION USING PSO FOR PREDICTING SEDIMENT TRANSPORT IN SEWERS
The flow in sewers is a complete three phase flow (air, water and sediment). The mechanism of sediment transport in sewers is very important. In other words, the passing flow must able to wash deposited sediments and the design should be done in an economic and optimized way. In this study, the sediment transport process in sewers is simulated using a hybrid model. In other words, using the Ada...
متن کاملApproximate Inference by Intersecting Semidefinite Bound and Local Polytope
Inference in probabilistic graphical models can be represented as a constrained optimization problem of a free-energy functional. Substantial research has been focused on the approximation of the constraint set, also known as the marginal polytope. This paper presents a novel inference algorithm that tightens and solves the optimization problem by intersecting the popular local polytope and the...
متن کامل